Eye gaze tracking is used in diagnosing and studying physiological and neurological disorders. It is also used as a research tool for understanding various cognitive functions such as vision and reading, in the areas of psychology and neurophysiology, and as a tool for studying effectiveness of marketing and advertising. In such off-line applications, eye gaze fixation data is often analyzed post-hoc, for example, to understand the object of a subject's interest. Eye gaze tracking is also used as an input in interactive applications. For example, in combination with a mouse or keyboard, eye gaze fixations can serve to disambiguate the selection of a target on a computer screen before movement of the mouse is initiated, or before a key is pressed. This allows for the use of a device such as a computer with little or no movement of the limbs; e.g., typing by looking at an on-screen keyboard layout. Further, eye gaze tracking enhances communication with a device through a speech production system, and enables control of a device remotely by looking at the device. Eye gaze tracking can also be used to enhance voice control of multiple devices by disambiguating voice commands. Finally, eye tracking can be used to evaluate effectiveness of visual designs, such as websites and cockpit instrument layouts. The applications of eye gaze tracking continue to grow, as does its importance as input separate from and complementary to the mouse and keyboard.
Wider integration of eye trackers into corporate, professional, and consumer systems requires that eye trackers be easy to use, affordable, and accurate, and less constrained by head and body movements of users. Unfortunately, current eye trackers leave much to be desired, as they are generally expensive, they require users to limit their head movements, and they require calibration, which is typically performed with help of a human operator. As such, current eye trackers are not suitable for applications in public places such as shopping malls or museums or as mass market products. Further, eye trackers with remote optics typically do not work if the user is farther than about 70 cm away from the camera, nor in point of regard tracking on surfaces larger than about 43 cm, thus practically restricting their use to applications such as desktop computers.
FIG. 3 shows the main components of a video-based eye tracking apparatus that utilizes remote optics. An infrared camera 305 is mounted near or below a screen 301, with one or more illuminators 304 placed near the axis 308 of the camera, which produce a bright pupil effect and glint in the eyes of a user, and an image processing facility that allows extraction of the pupil center and glint locations in an eye image. Alternatively, illuminators may be positioned off the optical camera axis, allowing a corneal glint but not a bright pupil. Alternatively, images with alternate on-axis and off-axis illumination are subtracted from one another, to isolate the pupil image. The location of the pupil and the glint in the eyes is typically determined by processing the camera image of the eye through various computer vision techniques.
Most eye tracking techniques require calibration in order to establish the parameters that describe the mapping between the eye coordinates as they appear in the camera image to the visual scene, or display coordinates. Many different calibration techniques exist, most of which involve knowledge of a detailed physiological model of the eye, eyeball radius and corneal curvature, the offset between optical and visual axis, head and eye location, the anterior chamber depth, as measured for a particular user, as well as the distance between the user and the camera, as measured throughout use. Some systems require that the location and angle of the camera is calibrated relative to the visual scene. To calibrate the system, the user is asked to look at a number of features (i.e., calibration points) in the visual scene, typically dots on a screen (for example, reference numerals 503 to 520 on FIG. 5), in sequence. This causes the subject's visual axis to align with the calibration point, which causes the pupil center in the camera image to appear away from the location of the camera glint in the eye, along a gaze vector with angle ρ, denoted reference numeral 523 in FIG. 5. The gaze vector will be different for each calibration point. The resulting set of gaze vectors, for each of which the corresponding point of gaze is known, is used to interpolate a random gaze vector 522, as measured by the eye tracker during operation, in respect of a point of regard 521 between calibration points. This is accomplished through an interpolation function that may include an (estimate of) a number of physiological parameters of the eye, accommodating for head position, screen position and size, and camera location and orientation, to adapt the gaze vector projection into the visual scene to the specific environmental circumstances, including the physiological properties of the subject's eye. This reduces the error in point of gaze projection to an acceptable level, which is typically within 1 degree of the visual angle. System calibration is typically only performed once per user. However, periodic recalibration may be required as environmental circumstances, such as ambient light levels, change.
A clear disadvantage of such prior calibration processes is that they require a continuous and directed effort on behalf of the subject. Such effort may not be available in infant or animal subjects, or in anonymous subjects that are required to use a gaze tracking system unsupervised in public places.
Amir et al. (U.S. Pat. No. 6,659,611, issued Dec. 9, 2003) discusses an approach to calibration in which an invisible test pattern is provided on a display intermittently throughout use. The test pattern may consist of infrared markers embedded in a known geometric formation in the screen. By gauging the warping present in the reflection of markers on the corneal surface, this technique aims to ascertain the mathematical transfer function that maps or interpolates a random gaze vector to arbitrary locations on a visual scene, typically a display. However, this technique has several disadvantages. Firstly, the mathematical warping function that models the curvature of the eye may be non-trivial. Secondly, the warping function may itself be warped non-linearly with different orientations of the eyeball, as the corneal sphere may not provide the same reflection at all orientations of the eye, requiring continuous measurement of the warping function. Thirdly, the accuracy of this method depends greatly on the accuracy of the underlying model of the eye, since the method itself provides no means of directly associating the location of a glint as reflected on the surface of the cornea, with that of the pupil center or optical axis. Finally, when a single camera is deployed, this technique requires the camera location and angle relative to the head and the screen to be known. Alternatively, it requires the use of a stereoscopic camera system.
U.S. Pat. No. 6,578,962, issued Jun. 17, 2003 to Amir et al., relates to another eye-gaze tracking method which requires two cameras, and requires relative positions and orientations of the cameras and the object being viewed by the subject to be known. This information is known from a one-time, user-dependent calibration of the system. Alternatively, when a single camera is deployed, this technique requires calibration of the radius of curvature of the cornea, and an estimate of the distance of the eye from the camera or the plane of the object being viewed by the subject.
U.S. Patent Application Publication No. 2004/0174496 A1, published on Sep. 9, 2004, relates to an eye gaze tracking method in which gaze is estimated from various calculated eye gaze parameters. This method uses mapping between the camera position and the image plane of the object being viewed, and the camera position must be known.